The Role of Referent Indicators in Tests of Measurement Invariance

In the social sciences, a considerable amount of research seeks to make comparisons between groups of people. These groups may be defined by nationality, culture, gender, race, or the same people at different points in time. Regardless of the substantive research questions, a key assumption is that the observed variables on which these comparisons are based, typically sets of items or scales, assess the groups in the same way. If items measure the same latent variables and are related to the latent variables in the same way, any observed differences across groups can be interpreted as true differences in the unobservable latent variables. The validity of this assumption is assessed by tests of measurement invariance; more precisely, measurement invariance (MI) can be considered the degree to which measurements conducted under different conditions yield equivalent measures of the same attributes (Horn & McArdle, 1992). If measurement invariance cannot be supported, differences between groups cannot be meaningfully interpreted. It is, therefore, critical that tests of measurement invariance produce valid, unambiguous results. The current study deals with an issue in confirmatory factor analytic (CFA) tests of measurement invariance, termed the “standardization problem” (Cheung & Rensvold, 1999; Rensvold & Cheung, 1998, 2001). The problem relates to the standardization procedures required with any use of CFA; however, the assumptions inherent in the procedure, when untenable, potentially compromise the validity of conclusions about MI. Thus, the goal of the current study is to determine the conditions under which researchers should be the most wary of tests of MI and those under which conclusions drawn from scale and item-level tests can safely be considered valid. Specifically, we focus on the role of the referent indicator (RI), which is the item chosen to provide a metric for the latent variable. The CFA Model and Measurement Invariance Tests The basic CFA model represents a linear relationship between a set of items and one or more latent constructs. Before conducting any CFA analyses, the researcher must ensure the model is identified (see Bollen, 1989 for a review) and that some scaling constraint is present in order to provide a metric for the latent variable. This can be done by assigning a value (typically 1.0) to either the variance of the latent factor or one of the item factor loadings, most commonly by the latter procedure (Bollen, 1989). When this option is chosen, the effect is such that scores on the latent variable are expressed in the scale of the selected item, the referent indicator (RI). Note that whether model identification is achieved via a referent indicator or standardized factor variances, model fit will not be affected. Typically, tests of MI compare the fit of a number of nested models in a procedure referred to as a likelihood ratio test (LRT). First, data from both groups is analyzed simultaneously, with the pattern of fixed and freed factor loadings held constant. This model serves as a baseline model to which more constrained models are then compared. Next, a test of “metric invariance” (Horn & McArdle, 1992) is conducted in which the established baseline model is compared to a model in which all factor loadings are constrained to be equal across group (i.e.,